Introduction to Computer Vision: Plant Seedlings Classification¶
Problem Statement¶
Context¶
In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
Objective¶
The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.
Data Dictionary¶
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
The dataset can be download from Olympus.
The data file names are:
- images.npy
- Labels.csv
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
- Black-grass
- Charlock
- Cleavers
- Common Chickweed
- Common Wheat
- Fat Hen
- Loose Silky-bent
- Maize
- Scentless Mayweed
- Shepherds Purse
- Small-flowered Cranesbill
- Sugar beet
Note: Please use GPU runtime on Google Colab to execute the code faster.¶
Importing necessary libraries¶
import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import math
import cv2
import seaborn as sns
# Tensorflow, Keras & Sklearn modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Model, Sequential, clone_model
from tensorflow.keras.layers import Dense,Dropout, Flatten, Conv2D, MaxPooling2D,BatchNormalization, GlobalAveragePooling2D
from tensorflow.keras.optimizers import Adam,SGD
from tensorflow.keras.applications import VGG16
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau, EarlyStopping
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix,classification_report,accuracy_score
from sklearn.preprocessing import LabelBinarizer
from sklearn.utils.class_weight import compute_class_weight
from sklearn.utils.validation import check_is_fitted
from sklearn.exceptions import NotFittedError
from sklearn.model_selection import train_test_split
# Display images using OpenCV
from google.colab.patches import cv2_imshow
import random
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
Loading the dataset¶
# Mounting google drive and initilizing path variable
from google.colab import drive
drive.mount("/content/drive")
path = '/content/drive/MyDrive/PGPAIML/Project-5/'
Mounted at /content/drive
# Load the image file of dataset
images = np.load(path+'images.npy')
# Load the labels file of dataset
labels = pd.read_csv(path+'labels.csv')
Data Overview¶
Understand the shape of the dataset¶
print(images.shape)
print(labels.shape)
(4750, 128, 128, 3) (4750, 1)
Observations
- We have 4750 labeled images as observations.
- The image size is 128 X 128 pixels and 3 channels.
Exploratory Data Analysis¶
- EDA is an important part of any project involving data.
- It is important to investigate and understand the data better before building a model with it.
Utility functions¶
# Function to display images in the provided range.
def show_images_from_data(images,labels, rows, cols, ranges):
num_classes=10
categories=np.unique(labels)
keys=dict(labels['Label'])
index = 0
fig = plt.figure(figsize=((3 * cols), (3.2 * rows)))
for i in range(cols):
for j in range(rows):
display_index = ranges[index]
index += 1
ax = fig.add_subplot(rows, cols, i * rows + j + 1)
ax.imshow(images[display_index])
ax.set_title(keys[display_index])
plt.show()
# Function to display labes count in a bar plot.
def display_label_count(labeles):
# Calculating label counts and percentages
label_counts = labels['Label'].value_counts()
label_percentages = (label_counts / len(labels) * 100).round(2)
label_summary = pd.DataFrame({'Count': label_counts, 'Percentage': label_percentages})
# Ploting the count plot with percentages
plt.figure(figsize=(12, 6))
sns.barplot(x=label_summary.index, y=label_summary['Count'], palette='viridis')
plt.xticks(rotation=90)
plt.title('Label Distribution with Percentages')
plt.xlabel('Labels')
plt.ylabel('Count')
# Annotate percentages on the bars
for i, p in enumerate(label_percentages):
plt.text(i, label_counts.iloc[i] + 5, f'{p}%', ha='center')
plt.tight_layout()
plt.show()
Visualizing Images & Labels¶
show_images_from_data(images, labels, 5, 5, range(0, 4749, 134))
display_label_count(labels)
Observations
- Looking at the images, domain knowledge and/or expertise are needed to classify them into proper categories, even for humans.
- The dataset apparently has an imbalanced distribution of labels.
- The most common label is
Loose Silky-bent(13.77%), followed closely byCommon Chickweed(12.86%). - The least represented labels are
Maize(4.65%),Common Wheat(4.65%), andShepherd's Purse(4.86%). - The class imbalance could bias the model toward the majority classes.
- Applying techniques like using class_weight or oversampling, undersampling, or data augmentation should be a good idea.
Decision
- I decided to Use the class weights approach during training, considering the below points
- No Data Duplication is needed
- Simpler, It integrates seamlessly into the
fit()method - Class weights dynamically adjust the importance of each class during training, which helps the model learn balanced features without manipulating the data.
Data Pre-Processing¶
Convert the BGR images to RGB images.¶
- As OpenCV Reads Images in BGR Format but TensorFlow/Keras expects images in RGB format, we need to convert BGR to RGB for consistency.
# Image format conversion
for i in range(len(images)):
image_bgr = images[i];
# Converting images using cv2
image_rgb = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
images[i] = image_rgb;
Resize the images¶
As the size of the images is large, it may be computationally expensive to train on these larger images; therefore, it is preferable to reduce the image size from 128 to 64.
Advantages of Reducing the Image Size:
- Smaller image sizes mean fewer input features, which significantly reduces the number of computations in convolutional and fully connected layers.
- This is especially useful when training from scratch on limited hardware or for faster experimentation.
Quicker Training:
- With smaller input dimensions, the training process is faster while still capturing essential patterns (as long as the reduction doesn't lose critical information).
Avoid Overfitting:
- Smaller images can help prevent overfitting by limiting the model’s ability to memorize unnecessary details.
images_resized = np.array([cv2.resize(img, (64, 64)) for img in images])
random_number = random.randint(1, 4700)
plt.imshow(images[random_number]);
plt.imshow(images_resized[random_number]);
Observations
To ensure that resizing to 64x64 does not result in a significant loss of information in the images, I displayed the same image above in both sizes. The observations show no significant distortions, though the resolution is affected. It is a good sign to proceed with the reduced image sizes.
Data Preparation for Modeling¶
Split the dataset
# Splitting data into Train (70%) and Temp (30%)
x_train, x_temp, y_train, y_temp = train_test_split(
images_resized, labels, test_size=0.3, random_state=1, stratify=labels
)
# Splitting Temp into Validation (50% of Temp -> 15% overall) and Test (50% of Temp -> 15% overall)
x_val, x_test, y_val, y_test = train_test_split(
x_temp, y_temp, test_size=0.5, random_state=1, stratify=y_temp
)
# Printing the shapes
print(f"Training Set: X={x_train.shape}, Y={y_train.shape}")
print(f"Validation Set: X={x_val.shape}, Y={y_val.shape}")
print(f"Test Set: X={x_test.shape}, Y={y_test.shape}")
Training Set: X=(3325, 64, 64, 3), Y=(3325, 1) Validation Set: X=(712, 64, 64, 3), Y=(712, 1) Test Set: X=(713, 64, 64, 3), Y=(713, 1)
Encode the target labels¶
# Initializing
label_binarizer = LabelBinarizer()
# Fit and transform
y_encoded = label_binarizer.fit_transform(labels['Label'])
# Verifying the shape
print("Encoded Labels Shape:", y_encoded.shape)
print("Sample Encoded Labels:\n", y_encoded[:5])
# Splitting data again after encoding if needed
y_train_encoded = label_binarizer.transform(y_train)
y_val_encoded = label_binarizer.transform(y_val)
y_test_encoded = label_binarizer.transform(y_test)
Encoded Labels Shape: (4750, 12) Sample Encoded Labels: [[0 0 0 0 0 0 0 0 0 0 1 0] [0 0 0 0 0 0 0 0 0 0 1 0] [0 0 0 0 0 0 0 0 0 0 1 0] [0 0 0 0 0 0 0 0 0 0 1 0] [0 0 0 0 0 0 0 0 0 0 1 0]]
Data Normalization¶
# Converting the image data to float and normalize pixel values
x_train_normalized = x_train.astype('float32') / 255.0
x_val_normalized = x_val.astype('float32') / 255.0
x_test_normalized = x_test.astype('float32') / 255.0
# Verifying normalization by checking min and max values
print(f"Training data range: {x_train_normalized.min()} to {x_train_normalized.max()}")
print(f"Validation data range: {x_val_normalized.min()} to {x_val_normalized.max()}")
print(f"Test data range: {x_test_normalized.min()} to {x_test_normalized.max()}")
Training data range: 0.0 to 1.0 Validation data range: 0.0 to 1.0 Test data range: 0.0 to 1.0
Model Building¶
Utility functions - Common Codes and Model Evaluation functions¶
# COOMON-CODE --> For Calculating Class Weights
y_train_labels = np.argmax(y_train_encoded, axis=1) # y_train_encoded is one-hot encoding
# Extract unique class labels
class_labels = np.unique(y_train_labels)
class_weights = compute_class_weight('balanced', classes=class_labels, y=y_train_labels)
class_weights_dict = {i: weight for i, weight in enumerate(class_weights)}
print("Computed Class Weights:", class_weights_dict)
Computed Class Weights: {0: 1.5058876811594204, 1: 1.014957264957265, 2: 1.378524046434494, 3: 0.6473909657320872, 4: 1.7876344086021505, 5: 0.8345883534136547, 6: 0.6049854439592431, 7: 1.7876344086021505, 8: 0.7675438596491229, 9: 1.7103909465020577, 10: 0.7985110470701249, 11: 1.0300495662949194}
# Function for setting seed for reproducibility & clearing session
def reset_environment(seed=1):
backend.clear_session()
np.random.seed(seed)
random.seed(seed)
tf.random.set_seed(seed)
tf.random.set_seed(seed)
tf.keras.utils.set_random_seed(seed)
# tf.config.experimental.enable_op_determinism() <-- this does not work when using GPU in Google CoLab
# Display loss plot for provided training history
def plot_loss(his):
plt.plot(his.history['loss'])
plt.plot(his.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
# Display accuracy plot for provided training history
def plot_accuracy(his):
plt.plot(his.history['accuracy'])
plt.plot(his.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
# Function to display multi-class classification confusion matrix
def plot_confusion_matrix_and_report(model, x, y):
y_pred = model.predict(x)
# Calculate confusion matrix
# Obtaining the categorical values
y_pred_arg = np.argmax(y_pred, axis=1)
y_arg = np.argmax(y, axis=1)
# Plotting the Confusion Matrix
cm = tf.math.confusion_matrix(y_arg, y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
cm,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(label_binarizer.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(label_binarizer.classes_),rotation=20)
plt.show()
cr = classification_report(y_arg, y_pred_arg)
print(cr)
Initial Model - first iteration¶
# Building the model
model1 = Sequential([
# First Conv2D layer
Conv2D(filters=32, kernel_size=(3, 3), activation='relu', padding='same', input_shape=(64, 64, 3)),
# Second Conv2D layer
Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same'),
# First MaxPooling layer
MaxPooling2D(pool_size=(2, 2)),
# Third Conv2D layer
Conv2D(filters=128, kernel_size=(3, 3), activation='relu', padding='same'),
# Second MaxPooling layer
MaxPooling2D(pool_size=(2, 2)),
# Flatten the output
Flatten(),
# First Dense layer with 16 neurons
Dense(16, activation='relu'),
# Output layer with 12 neurons for 12 classes
Dense(12, activation='softmax')
])
# Compile the model
model1.compile(optimizer=Adam(),
loss='categorical_crossentropy',
metrics=['accuracy'])
# Print the model summary
model1.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ conv2d (Conv2D) │ (None, 64, 64, 32) │ 896 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_1 (Conv2D) │ (None, 64, 64, 64) │ 18,496 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d (MaxPooling2D) │ (None, 32, 32, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_2 (Conv2D) │ (None, 32, 32, 128) │ 73,856 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_1 (MaxPooling2D) │ (None, 16, 16, 128) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ flatten (Flatten) │ (None, 32768) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense (Dense) │ (None, 16) │ 524,304 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 12) │ 204 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 617,756 (2.36 MB)
Trainable params: 617,756 (2.36 MB)
Non-trainable params: 0 (0.00 B)
Fitting the model on the train data
# Resetting the environment default seed=1
reset_environment()
# Fit the model
history1 = model1.fit(
x_train_normalized,
y_train_encoded,
epochs=50,
validation_data=(x_val_normalized,y_val_encoded),
batch_size=32,
class_weight=class_weights_dict,
verbose=2
)
Epoch 1/50 104/104 - 9s - 89ms/step - accuracy: 0.0586 - loss: 2.4879 - val_accuracy: 0.1081 - val_loss: 2.4846 Epoch 2/50 104/104 - 4s - 35ms/step - accuracy: 0.1684 - loss: 2.3858 - val_accuracy: 0.2191 - val_loss: 2.2268 Epoch 3/50 104/104 - 2s - 24ms/step - accuracy: 0.1808 - loss: 2.1848 - val_accuracy: 0.1910 - val_loss: 2.0911 Epoch 4/50 104/104 - 1s - 12ms/step - accuracy: 0.2033 - loss: 2.0571 - val_accuracy: 0.2331 - val_loss: 1.9894 Epoch 5/50 104/104 - 1s - 12ms/step - accuracy: 0.2376 - loss: 1.9568 - val_accuracy: 0.2669 - val_loss: 1.9061 Epoch 6/50 104/104 - 1s - 13ms/step - accuracy: 0.2514 - loss: 1.8740 - val_accuracy: 0.2992 - val_loss: 1.8571 Epoch 7/50 104/104 - 2s - 23ms/step - accuracy: 0.2863 - loss: 1.8027 - val_accuracy: 0.3343 - val_loss: 1.8168 Epoch 8/50 104/104 - 1s - 12ms/step - accuracy: 0.3173 - loss: 1.7535 - val_accuracy: 0.3427 - val_loss: 1.8103 Epoch 9/50 104/104 - 1s - 12ms/step - accuracy: 0.3245 - loss: 1.7212 - val_accuracy: 0.3539 - val_loss: 1.8127 Epoch 10/50 104/104 - 1s - 12ms/step - accuracy: 0.3432 - loss: 1.6857 - val_accuracy: 0.3413 - val_loss: 1.8111 Epoch 11/50 104/104 - 1s - 12ms/step - accuracy: 0.3573 - loss: 1.6636 - val_accuracy: 0.3539 - val_loss: 1.8432 Epoch 12/50 104/104 - 1s - 12ms/step - accuracy: 0.3711 - loss: 1.6506 - val_accuracy: 0.3315 - val_loss: 1.8283 Epoch 13/50 104/104 - 1s - 12ms/step - accuracy: 0.3696 - loss: 1.6189 - val_accuracy: 0.3441 - val_loss: 1.7755 Epoch 14/50 104/104 - 1s - 12ms/step - accuracy: 0.3865 - loss: 1.5723 - val_accuracy: 0.3469 - val_loss: 1.7939 Epoch 15/50 104/104 - 1s - 13ms/step - accuracy: 0.4024 - loss: 1.5228 - val_accuracy: 0.3553 - val_loss: 1.8043 Epoch 16/50 104/104 - 1s - 13ms/step - accuracy: 0.4084 - loss: 1.5085 - val_accuracy: 0.3497 - val_loss: 1.7758 Epoch 17/50 104/104 - 1s - 13ms/step - accuracy: 0.4102 - loss: 1.4640 - val_accuracy: 0.3301 - val_loss: 1.8260 Epoch 18/50 104/104 - 2s - 24ms/step - accuracy: 0.4274 - loss: 1.4278 - val_accuracy: 0.3343 - val_loss: 1.8257 Epoch 19/50 104/104 - 1s - 12ms/step - accuracy: 0.4241 - loss: 1.4066 - val_accuracy: 0.3469 - val_loss: 1.8382 Epoch 20/50 104/104 - 1s - 13ms/step - accuracy: 0.4370 - loss: 1.3758 - val_accuracy: 0.3525 - val_loss: 1.9152 Epoch 21/50 104/104 - 2s - 24ms/step - accuracy: 0.4349 - loss: 1.3601 - val_accuracy: 0.3301 - val_loss: 1.9733 Epoch 22/50 104/104 - 1s - 13ms/step - accuracy: 0.4358 - loss: 1.3393 - val_accuracy: 0.3315 - val_loss: 1.9477 Epoch 23/50 104/104 - 3s - 25ms/step - accuracy: 0.4361 - loss: 1.3295 - val_accuracy: 0.3399 - val_loss: 1.9014 Epoch 24/50 104/104 - 1s - 13ms/step - accuracy: 0.4355 - loss: 1.3604 - val_accuracy: 0.3497 - val_loss: 1.8440 Epoch 25/50 104/104 - 1s - 13ms/step - accuracy: 0.4740 - loss: 1.2472 - val_accuracy: 0.3553 - val_loss: 1.9416 Epoch 26/50 104/104 - 1s - 12ms/step - accuracy: 0.4953 - loss: 1.1875 - val_accuracy: 0.3539 - val_loss: 2.0510 Epoch 27/50 104/104 - 1s - 13ms/step - accuracy: 0.5089 - loss: 1.1347 - val_accuracy: 0.3441 - val_loss: 2.1178 Epoch 28/50 104/104 - 2s - 24ms/step - accuracy: 0.5191 - loss: 1.0999 - val_accuracy: 0.3469 - val_loss: 2.1812 Epoch 29/50 104/104 - 1s - 12ms/step - accuracy: 0.5338 - loss: 1.0761 - val_accuracy: 0.3525 - val_loss: 2.2380 Epoch 30/50 104/104 - 1s - 12ms/step - accuracy: 0.5374 - loss: 1.0744 - val_accuracy: 0.3553 - val_loss: 2.2195 Epoch 31/50 104/104 - 3s - 24ms/step - accuracy: 0.5230 - loss: 1.0672 - val_accuracy: 0.3680 - val_loss: 2.3535 Epoch 32/50 104/104 - 3s - 25ms/step - accuracy: 0.5293 - loss: 1.0764 - val_accuracy: 0.3399 - val_loss: 2.4034 Epoch 33/50 104/104 - 2s - 24ms/step - accuracy: 0.5377 - loss: 1.0444 - val_accuracy: 0.3652 - val_loss: 2.1508 Epoch 34/50 104/104 - 1s - 12ms/step - accuracy: 0.5429 - loss: 1.0224 - val_accuracy: 0.3455 - val_loss: 2.1989 Epoch 35/50 104/104 - 2s - 24ms/step - accuracy: 0.5765 - loss: 0.9808 - val_accuracy: 0.3483 - val_loss: 2.3913 Epoch 36/50 104/104 - 1s - 13ms/step - accuracy: 0.5774 - loss: 0.9716 - val_accuracy: 0.3652 - val_loss: 2.4277 Epoch 37/50 104/104 - 1s - 12ms/step - accuracy: 0.5829 - loss: 0.9731 - val_accuracy: 0.3722 - val_loss: 2.4276 Epoch 38/50 104/104 - 1s - 12ms/step - accuracy: 0.5699 - loss: 0.9800 - val_accuracy: 0.3581 - val_loss: 2.4464 Epoch 39/50 104/104 - 1s - 13ms/step - accuracy: 0.5771 - loss: 0.9834 - val_accuracy: 0.3343 - val_loss: 2.5602 Epoch 40/50 104/104 - 2s - 24ms/step - accuracy: 0.5994 - loss: 0.9377 - val_accuracy: 0.3427 - val_loss: 2.7252 Epoch 41/50 104/104 - 1s - 12ms/step - accuracy: 0.5997 - loss: 0.9203 - val_accuracy: 0.3497 - val_loss: 2.5780 Epoch 42/50 104/104 - 1s - 12ms/step - accuracy: 0.6298 - loss: 0.8677 - val_accuracy: 0.3455 - val_loss: 2.8309 Epoch 43/50 104/104 - 1s - 12ms/step - accuracy: 0.6523 - loss: 0.8475 - val_accuracy: 0.3553 - val_loss: 2.9278 Epoch 44/50 104/104 - 1s - 12ms/step - accuracy: 0.6415 - loss: 0.8627 - val_accuracy: 0.3722 - val_loss: 2.8745 Epoch 45/50 104/104 - 1s - 12ms/step - accuracy: 0.6235 - loss: 0.9178 - val_accuracy: 0.3764 - val_loss: 2.9087 Epoch 46/50 104/104 - 1s - 12ms/step - accuracy: 0.6042 - loss: 0.9236 - val_accuracy: 0.3553 - val_loss: 2.8286 Epoch 47/50 104/104 - 1s - 12ms/step - accuracy: 0.6325 - loss: 0.8793 - val_accuracy: 0.3848 - val_loss: 2.6239 Epoch 48/50 104/104 - 1s - 12ms/step - accuracy: 0.6493 - loss: 0.8115 - val_accuracy: 0.3497 - val_loss: 2.9943 Epoch 49/50 104/104 - 3s - 25ms/step - accuracy: 0.6725 - loss: 0.7891 - val_accuracy: 0.3750 - val_loss: 2.9498 Epoch 50/50 104/104 - 2s - 23ms/step - accuracy: 0.6761 - loss: 0.7900 - val_accuracy: 0.3778 - val_loss: 2.8940
Model Evaluation
plot_loss(history1)
plot_accuracy(history1)
plot_confusion_matrix_and_report(model1, x_val_normalized, y_val_encoded)
23/23 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step
precision recall f1-score support
0 0.39 0.42 0.40 40
1 0.42 0.37 0.39 59
2 0.29 0.26 0.27 43
3 0.76 0.48 0.59 91
4 0.18 0.33 0.23 33
5 0.42 0.34 0.38 71
6 0.44 0.33 0.38 98
7 0.16 0.27 0.20 33
8 0.36 0.39 0.38 77
9 0.16 0.29 0.20 35
10 0.61 0.58 0.60 74
11 0.29 0.28 0.28 58
accuracy 0.38 712
macro avg 0.37 0.36 0.36 712
weighted avg 0.42 0.38 0.39 712
Observations¶
Loss and Accuracy Plots:
- The training loss consistently decreases, and training accuracy steadily increases, indicating the model is learning the training data well. Training accuracy reaches ~70% by the end of 50 epochs.
- Validation loss decreases initially but starts to oscillate and increase after ~25 epochs, indicating overfitting. Validation accuracy plateaus at ~38%, showing poor generalization to the validation set.
Confusion Matrix:
- The model shows significant confusion among classes, especially for classes like:
Loose Silky-bentis predicted asFat HenorCommon Chickweedfrequently.Scentless Mayweedis confused withShepherds PurseandSmall-flowered Cranesbill.
- Some classes (e.g.,
Common ChickweedandLoose Silky-bent) are predicted relatively better, but others have poor recall.
Classification Report:
- Average precision: 0.42 (low, indicating many false positives).
- Average recall: 0.36 (low, indicating many false negatives).
- Weighted F1-score: 0.39, showing imbalanced performance across classes.
- Certain classes like
CleaversandSmall-flowered Cranesbillperform relatively better but are still far from optimal. - Some classes (e.g.,
Scentless Mayweed,Loose Silky-bent) have very low recall and F1-scores.
Issues with this Model:
Overfitting: Training accuracy is significantly higher than validation accuracy, and validation loss increases over epochs. The model is overfitting to the training data.
Class Imbalance: The model struggles to handle class imbalance effectively, despite using
class_weight. This is evident in the poor recall and precision for minority classes.Model Complexity: The current architecture may not have sufficient capacity to capture complex features in the dataset. A deeper model with more layers or filters may help.
No Regularization: No techniques like Batch Normalization or Dropout are applied, leading to overfitting.
Model Performance Improvement¶
Second Iteration¶
Improvement Opinions
- Add BatchNormalization after convolutional layers to stabilize learning.
- Add more convolutional layers or increase the number of filters in existing layers to allow the model to capture more complex patterns
- Use a learning rate of 1e-4 instead of the default 1e-3 to allow finer adjustments to weights.
# Building the second model
model2 = Sequential([
# First Conv2D layer
Conv2D(filters=32, kernel_size=(3, 3), activation='relu', padding='same', input_shape=(64, 64, 3)),
# Second Conv2D layer
Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same'),
# First MaxPooling layer
MaxPooling2D(pool_size=(2, 2)),
# Third Conv2D layer <-- Added New
Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same'),
# Fourth Conv2D layer
Conv2D(filters=128, kernel_size=(3, 3), activation='relu', padding='same'),
# Second MaxPooling layer
MaxPooling2D(pool_size=(2, 2)),
# BatchNormalization <-- Added New
BatchNormalization(),
# Flatten the output
Flatten(),
# First Dense layer with 16 neurons
Dense(16, activation='relu'),
# Output layer with 12 neurons for 12 classes
Dense(12, activation='softmax')
])
# Compile the model
model2.compile(optimizer=Adam(learning_rate=1e-4),# Reduced learning_rate=0.0005 <-- Added new
loss='categorical_crossentropy',
metrics=['accuracy'])
# Print the model summary
model2.summary()
Model: "sequential_1"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ conv2d_4 (Conv2D) │ (None, 64, 64, 32) │ 896 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_5 (Conv2D) │ (None, 64, 64, 64) │ 18,496 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_2 (MaxPooling2D) │ (None, 32, 32, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_6 (Conv2D) │ (None, 32, 32, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_7 (Conv2D) │ (None, 32, 32, 128) │ 73,856 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_3 (MaxPooling2D) │ (None, 16, 16, 128) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ batch_normalization_1 │ (None, 16, 16, 128) │ 512 │ │ (BatchNormalization) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ flatten_1 (Flatten) │ (None, 32768) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_2 (Dense) │ (None, 16) │ 524,304 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_3 (Dense) │ (None, 12) │ 204 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 655,196 (2.50 MB)
Trainable params: 654,940 (2.50 MB)
Non-trainable params: 256 (1.00 KB)
# Resetting the environment default seed=1
reset_environment()
# Fit the model
history2 = model2.fit(
x_train_normalized,
y_train_encoded,
epochs=50,
validation_data=(x_val_normalized,y_val_encoded),
batch_size=32,
class_weight=class_weights_dict,
verbose=2
)
Epoch 1/50 104/104 - 10s - 97ms/step - accuracy: 0.2162 - loss: 2.2047 - val_accuracy: 0.1854 - val_loss: 2.4180 Epoch 2/50 104/104 - 2s - 15ms/step - accuracy: 0.3874 - loss: 1.6098 - val_accuracy: 0.2992 - val_loss: 2.2801 Epoch 3/50 104/104 - 2s - 15ms/step - accuracy: 0.5531 - loss: 1.2259 - val_accuracy: 0.4733 - val_loss: 2.0092 Epoch 4/50 104/104 - 2s - 15ms/step - accuracy: 0.6788 - loss: 0.9448 - val_accuracy: 0.5506 - val_loss: 1.6380 Epoch 5/50 104/104 - 2s - 24ms/step - accuracy: 0.7603 - loss: 0.7325 - val_accuracy: 0.5772 - val_loss: 1.2894 Epoch 6/50 104/104 - 3s - 24ms/step - accuracy: 0.8352 - loss: 0.5630 - val_accuracy: 0.6081 - val_loss: 1.1665 Epoch 7/50 104/104 - 2s - 24ms/step - accuracy: 0.8722 - loss: 0.4467 - val_accuracy: 0.6362 - val_loss: 1.1952 Epoch 8/50 104/104 - 1s - 14ms/step - accuracy: 0.9023 - loss: 0.3515 - val_accuracy: 0.5913 - val_loss: 1.4839 Epoch 9/50 104/104 - 2s - 15ms/step - accuracy: 0.9209 - loss: 0.2861 - val_accuracy: 0.5197 - val_loss: 1.9568 Epoch 10/50 104/104 - 3s - 25ms/step - accuracy: 0.9426 - loss: 0.2348 - val_accuracy: 0.5815 - val_loss: 1.6466 Epoch 11/50 104/104 - 2s - 15ms/step - accuracy: 0.9570 - loss: 0.1802 - val_accuracy: 0.5295 - val_loss: 2.0402 Epoch 12/50 104/104 - 2s - 24ms/step - accuracy: 0.9753 - loss: 0.1371 - val_accuracy: 0.5492 - val_loss: 1.8564 Epoch 13/50 104/104 - 3s - 25ms/step - accuracy: 0.9868 - loss: 0.1007 - val_accuracy: 0.5899 - val_loss: 1.5816 Epoch 14/50 104/104 - 3s - 24ms/step - accuracy: 0.9931 - loss: 0.0711 - val_accuracy: 0.6657 - val_loss: 1.3870 Epoch 15/50 104/104 - 2s - 15ms/step - accuracy: 0.9964 - loss: 0.0544 - val_accuracy: 0.6770 - val_loss: 1.2908 Epoch 16/50 104/104 - 3s - 25ms/step - accuracy: 0.9982 - loss: 0.0427 - val_accuracy: 0.6910 - val_loss: 1.3511 Epoch 17/50 104/104 - 2s - 24ms/step - accuracy: 0.9982 - loss: 0.0342 - val_accuracy: 0.6713 - val_loss: 1.3461 Epoch 18/50 104/104 - 1s - 14ms/step - accuracy: 0.9991 - loss: 0.0261 - val_accuracy: 0.6840 - val_loss: 1.3300 Epoch 19/50 104/104 - 3s - 25ms/step - accuracy: 0.9991 - loss: 0.0202 - val_accuracy: 0.6882 - val_loss: 1.3796 Epoch 20/50 104/104 - 2s - 24ms/step - accuracy: 0.9997 - loss: 0.0167 - val_accuracy: 0.6826 - val_loss: 1.3975 Epoch 21/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0140 - val_accuracy: 0.6784 - val_loss: 1.4246 Epoch 22/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0119 - val_accuracy: 0.6784 - val_loss: 1.4164 Epoch 23/50 104/104 - 2s - 24ms/step - accuracy: 0.9997 - loss: 0.0107 - val_accuracy: 0.6798 - val_loss: 1.4395 Epoch 24/50 104/104 - 3s - 25ms/step - accuracy: 0.9997 - loss: 0.0092 - val_accuracy: 0.6812 - val_loss: 1.4460 Epoch 25/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0081 - val_accuracy: 0.6826 - val_loss: 1.4377 Epoch 26/50 104/104 - 3s - 24ms/step - accuracy: 0.9997 - loss: 0.0072 - val_accuracy: 0.6868 - val_loss: 1.4278 Epoch 27/50 104/104 - 3s - 25ms/step - accuracy: 0.9997 - loss: 0.0065 - val_accuracy: 0.6910 - val_loss: 1.4128 Epoch 28/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0059 - val_accuracy: 0.6966 - val_loss: 1.4098 Epoch 29/50 104/104 - 2s - 16ms/step - accuracy: 0.9997 - loss: 0.0056 - val_accuracy: 0.7008 - val_loss: 1.4114 Epoch 30/50 104/104 - 2s - 23ms/step - accuracy: 0.9997 - loss: 0.0049 - val_accuracy: 0.7051 - val_loss: 1.4172 Epoch 31/50 104/104 - 3s - 24ms/step - accuracy: 0.9997 - loss: 0.0044 - val_accuracy: 0.7051 - val_loss: 1.4150 Epoch 32/50 104/104 - 3s - 24ms/step - accuracy: 0.9997 - loss: 0.0040 - val_accuracy: 0.7135 - val_loss: 1.4217 Epoch 33/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0037 - val_accuracy: 0.7205 - val_loss: 1.4312 Epoch 34/50 104/104 - 3s - 25ms/step - accuracy: 0.9997 - loss: 0.0034 - val_accuracy: 0.7191 - val_loss: 1.4418 Epoch 35/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0032 - val_accuracy: 0.7219 - val_loss: 1.4453 Epoch 36/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0029 - val_accuracy: 0.7149 - val_loss: 1.4582 Epoch 37/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0027 - val_accuracy: 0.7135 - val_loss: 1.4672 Epoch 38/50 104/104 - 3s - 24ms/step - accuracy: 0.9976 - loss: 0.0091 - val_accuracy: 0.4944 - val_loss: 3.7317 Epoch 39/50 104/104 - 2s - 15ms/step - accuracy: 0.9347 - loss: 0.2042 - val_accuracy: 0.5758 - val_loss: 2.0891 Epoch 40/50 104/104 - 2s - 24ms/step - accuracy: 0.9555 - loss: 0.1440 - val_accuracy: 0.5028 - val_loss: 3.1505 Epoch 41/50 104/104 - 2s - 15ms/step - accuracy: 0.9904 - loss: 0.0384 - val_accuracy: 0.6461 - val_loss: 1.8803 Epoch 42/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0077 - val_accuracy: 0.6938 - val_loss: 1.7021 Epoch 43/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0042 - val_accuracy: 0.6994 - val_loss: 1.6878 Epoch 44/50 104/104 - 1s - 14ms/step - accuracy: 0.9997 - loss: 0.0033 - val_accuracy: 0.7037 - val_loss: 1.7050 Epoch 45/50 104/104 - 1s - 14ms/step - accuracy: 0.9997 - loss: 0.0028 - val_accuracy: 0.7022 - val_loss: 1.7121 Epoch 46/50 104/104 - 1s - 14ms/step - accuracy: 0.9997 - loss: 0.0026 - val_accuracy: 0.6994 - val_loss: 1.7112 Epoch 47/50 104/104 - 3s - 25ms/step - accuracy: 0.9997 - loss: 0.0023 - val_accuracy: 0.7022 - val_loss: 1.7150 Epoch 48/50 104/104 - 2s - 24ms/step - accuracy: 0.9997 - loss: 0.0021 - val_accuracy: 0.6994 - val_loss: 1.7178 Epoch 49/50 104/104 - 3s - 25ms/step - accuracy: 0.9997 - loss: 0.0020 - val_accuracy: 0.7008 - val_loss: 1.7151 Epoch 50/50 104/104 - 2s - 15ms/step - accuracy: 0.9997 - loss: 0.0019 - val_accuracy: 0.6952 - val_loss: 1.7200
plot_loss(history2)
plot_accuracy(history2)
plot_confusion_matrix_and_report(model2, x_val_normalized, y_val_encoded)
23/23 ━━━━━━━━━━━━━━━━━━━━ 1s 17ms/step
precision recall f1-score support
0 0.42 0.45 0.43 40
1 0.80 0.80 0.80 59
2 0.80 0.77 0.79 43
3 0.80 0.85 0.82 91
4 0.59 0.48 0.53 33
5 0.66 0.59 0.62 71
6 0.60 0.80 0.69 98
7 0.73 0.67 0.70 33
8 0.68 0.61 0.64 77
9 0.58 0.40 0.47 35
10 0.85 0.92 0.88 74
11 0.66 0.57 0.61 58
accuracy 0.70 712
macro avg 0.68 0.66 0.67 712
weighted avg 0.70 0.70 0.69 712
Observations¶
Loss and Accuracy Plots:
- Training loss decreases consistently to near 0.0, and training accuracy reaches ~1.0. This shows that the model is fitting the training data very well, indicating a potential risk of overfitting.
- Validation loss decreases initially but fluctuates and rises after around 25 epochs. Validation accuracy stabilizes at ~70%, which is a significant improvement compared to the first model.
Confusion Matrix:
- The confusion matrix shows a better spread of predictions across classes compared to the first model.
- Classes such as
Loose Silky-bent,Common Chickweed, andSmall-flowered Cranesbillare predicted more accurately. Misclassifications are still present but significantly reduced. - Some confusions persist, for example:
Loose Silky-bentis occasionally misclassified asBlack-grass.Scentless Mayweedis confused withShepherds Purse.
Classification Report:
- Precision: 0.68 (macro avg), up from 0.42 in the first model.
- Recall: 0.66 (macro avg), up from 0.36 in the first model.
- Weighted F1-score: 0.69, showing much better balance across classes.
- Most classes show significant improvement in precision and recall.
Cleavers,Loose Silky-bent, andSmall-flowered Cranesbillhave F1-scores over 0.70, while underrepresented classes likeMaizeandShepherds Pursestill need improvement.
Improvements Made:
Model Complexity: Shifting to fourth
Conv2Dlayer by adding extraConv2Dwith 64 filters improved the model's feature extraction capability.Batch Normalization: The addition of
BatchNormalizationstabilized training and helped generalization, contributing to better validation performance.Learning Rate Adjustment: Lowering the learning rate to
1e-4allowed the model to converge more effectively, resulting in improved validation accuracy.
Still exists issues:
Overfitting: Training accuracy is significantly higher than validation accuracy, and validation loss fluctuates and increases after ~25 epochs. This indicates overfitting, despite the improvements.
Misclassifications: Certain confusions persist, especially for underrepresented or visually similar classes.
Class Imbalance: Class imbalance is still a challenge, as seen in the relatively lower performance for minority classes like
MaizeandShepherds Purse.
Third Iteration¶
Improvement Options
- Data Augmentation - Apply ImageDataGenerator to artificially increase the variability of training data and improve generalization.
- Add another dense layer before the output layer or increase neurons in the existing dense layer.
- Add Dropout layers after dense layers and convolutional layers to reduce overfitting.
# Creating & Fitting DataGenerator <-- Added new
augmented_datagen = ImageDataGenerator(
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True
)
augmented_datagen.fit(x_train_normalized)
# Building the third model
model3 = Sequential([
# First Conv2D layer
Conv2D(filters=32, kernel_size=(3, 3), activation='relu', padding='same', input_shape=(64, 64, 3)),
# Second Conv2D layer
Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same'),
# First MaxPooling layer
MaxPooling2D(pool_size=(2, 2)),
# Third Conv2D layer
Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same'),
# Fourth Conv2D layer
Conv2D(filters=128, kernel_size=(3, 3), activation='relu', padding='same'),
# Second MaxPooling layer
MaxPooling2D(pool_size=(2, 2)),
# First Dropout Layer<-- Added New
Dropout(0.25),
# BatchNormalization
BatchNormalization(),
# Flatten the output
Flatten(),
# First Dense layer with 16 neurons
Dense(16, activation='relu'),
# Second Dense layer with 32 neurons <-- Added New
Dense(32, activation='relu'),
# Second Dropout Layer<-- Added New
Dropout(0.25),
# Output layer with 12 neurons for 12 classes
Dense(12, activation='softmax')
])
# Compile the model
model3.compile(optimizer=Adam(learning_rate=1e-4),# Reduced learning_rate=0.0005
loss='categorical_crossentropy',
metrics=['accuracy'])
# Print the model summary
model3.summary()
Model: "sequential_1"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ conv2d_4 (Conv2D) │ (None, 64, 64, 32) │ 896 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_5 (Conv2D) │ (None, 64, 64, 64) │ 18,496 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_2 (MaxPooling2D) │ (None, 32, 32, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_6 (Conv2D) │ (None, 32, 32, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_7 (Conv2D) │ (None, 32, 32, 128) │ 73,856 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_3 (MaxPooling2D) │ (None, 16, 16, 128) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout_2 (Dropout) │ (None, 16, 16, 128) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ batch_normalization_1 │ (None, 16, 16, 128) │ 512 │ │ (BatchNormalization) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ flatten_1 (Flatten) │ (None, 32768) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_3 (Dense) │ (None, 16) │ 524,304 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_4 (Dense) │ (None, 32) │ 544 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout_3 (Dropout) │ (None, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_5 (Dense) │ (None, 12) │ 396 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 655,932 (2.50 MB)
Trainable params: 655,676 (2.50 MB)
Non-trainable params: 256 (1.00 KB)
# Resetting the environment default seed=1
reset_environment()
# Fit the model
history3 = model3.fit(
augmented_datagen.flow( # Data Augmentation <-- Added New
x_train_normalized,
y_train_encoded,
batch_size=32),
epochs=50,
validation_data=(x_val_normalized,
y_val_encoded),
class_weight=class_weights_dict,
verbose=2)
Epoch 1/50 104/104 - 25s - 238ms/step - accuracy: 0.0824 - loss: 2.4888 - val_accuracy: 0.0590 - val_loss: 2.4854 Epoch 2/50 104/104 - 24s - 233ms/step - accuracy: 0.0896 - loss: 2.4392 - val_accuracy: 0.1742 - val_loss: 2.4669 Epoch 3/50 104/104 - 4s - 38ms/step - accuracy: 0.1308 - loss: 2.3840 - val_accuracy: 0.1236 - val_loss: 2.4258 Epoch 4/50 104/104 - 6s - 59ms/step - accuracy: 0.2066 - loss: 2.2591 - val_accuracy: 0.2177 - val_loss: 2.3125 Epoch 5/50 104/104 - 5s - 45ms/step - accuracy: 0.2941 - loss: 2.1051 - val_accuracy: 0.2542 - val_loss: 2.0872 Epoch 6/50 104/104 - 4s - 38ms/step - accuracy: 0.3398 - loss: 1.9675 - val_accuracy: 0.3975 - val_loss: 1.8269 Epoch 7/50 104/104 - 4s - 42ms/step - accuracy: 0.3603 - loss: 1.8671 - val_accuracy: 0.4874 - val_loss: 1.6008 Epoch 8/50 104/104 - 5s - 52ms/step - accuracy: 0.3765 - loss: 1.8125 - val_accuracy: 0.4747 - val_loss: 1.5606 Epoch 9/50 104/104 - 10s - 91ms/step - accuracy: 0.4006 - loss: 1.7406 - val_accuracy: 0.4789 - val_loss: 1.3795 Epoch 10/50 104/104 - 5s - 49ms/step - accuracy: 0.4256 - loss: 1.6812 - val_accuracy: 0.4944 - val_loss: 1.4736 Epoch 11/50 104/104 - 10s - 98ms/step - accuracy: 0.4466 - loss: 1.6049 - val_accuracy: 0.5576 - val_loss: 1.2525 Epoch 12/50 104/104 - 5s - 47ms/step - accuracy: 0.4550 - loss: 1.5417 - val_accuracy: 0.6250 - val_loss: 1.1483 Epoch 13/50 104/104 - 4s - 38ms/step - accuracy: 0.4794 - loss: 1.4777 - val_accuracy: 0.4382 - val_loss: 1.5128 Epoch 14/50 104/104 - 6s - 61ms/step - accuracy: 0.4803 - loss: 1.4814 - val_accuracy: 0.5787 - val_loss: 1.1904 Epoch 15/50 104/104 - 4s - 41ms/step - accuracy: 0.4977 - loss: 1.4364 - val_accuracy: 0.6320 - val_loss: 1.0410 Epoch 16/50 104/104 - 4s - 37ms/step - accuracy: 0.5071 - loss: 1.3926 - val_accuracy: 0.5744 - val_loss: 1.2002 Epoch 17/50 104/104 - 5s - 45ms/step - accuracy: 0.5041 - loss: 1.3596 - val_accuracy: 0.6320 - val_loss: 1.0475 Epoch 18/50 104/104 - 5s - 47ms/step - accuracy: 0.5275 - loss: 1.3164 - val_accuracy: 0.6138 - val_loss: 1.0857 Epoch 19/50 104/104 - 5s - 45ms/step - accuracy: 0.5389 - loss: 1.3017 - val_accuracy: 0.6573 - val_loss: 0.9416 Epoch 20/50 104/104 - 5s - 48ms/step - accuracy: 0.5576 - loss: 1.2704 - val_accuracy: 0.5885 - val_loss: 1.0916 Epoch 21/50 104/104 - 4s - 43ms/step - accuracy: 0.5477 - loss: 1.2291 - val_accuracy: 0.6194 - val_loss: 1.0011 Epoch 22/50 104/104 - 4s - 37ms/step - accuracy: 0.5768 - loss: 1.2007 - val_accuracy: 0.6840 - val_loss: 0.9379 Epoch 23/50 104/104 - 8s - 80ms/step - accuracy: 0.5865 - loss: 1.1704 - val_accuracy: 0.6671 - val_loss: 0.9104 Epoch 24/50 104/104 - 4s - 38ms/step - accuracy: 0.5874 - loss: 1.1610 - val_accuracy: 0.6812 - val_loss: 0.9318 Epoch 25/50 104/104 - 4s - 39ms/step - accuracy: 0.6021 - loss: 1.1050 - val_accuracy: 0.6222 - val_loss: 1.0808 Epoch 26/50 104/104 - 6s - 56ms/step - accuracy: 0.6150 - loss: 1.0763 - val_accuracy: 0.4916 - val_loss: 1.4650 Epoch 27/50 104/104 - 4s - 37ms/step - accuracy: 0.6307 - loss: 1.0768 - val_accuracy: 0.7514 - val_loss: 0.7269 Epoch 28/50 104/104 - 5s - 49ms/step - accuracy: 0.6406 - loss: 1.0228 - val_accuracy: 0.7739 - val_loss: 0.7146 Epoch 29/50 104/104 - 7s - 63ms/step - accuracy: 0.6358 - loss: 1.0350 - val_accuracy: 0.6475 - val_loss: 1.0014 Epoch 30/50 104/104 - 10s - 93ms/step - accuracy: 0.6481 - loss: 0.9739 - val_accuracy: 0.5983 - val_loss: 1.1749 Epoch 31/50 104/104 - 5s - 47ms/step - accuracy: 0.6644 - loss: 0.9754 - val_accuracy: 0.7416 - val_loss: 0.6945 Epoch 32/50 104/104 - 4s - 37ms/step - accuracy: 0.6647 - loss: 0.9579 - val_accuracy: 0.7008 - val_loss: 0.8546 Epoch 33/50 104/104 - 4s - 40ms/step - accuracy: 0.6737 - loss: 0.9408 - val_accuracy: 0.6138 - val_loss: 1.0805 Epoch 34/50 104/104 - 6s - 56ms/step - accuracy: 0.6938 - loss: 0.8978 - val_accuracy: 0.7205 - val_loss: 0.7687 Epoch 35/50 104/104 - 4s - 38ms/step - accuracy: 0.6980 - loss: 0.8925 - val_accuracy: 0.7739 - val_loss: 0.6521 Epoch 36/50 104/104 - 4s - 42ms/step - accuracy: 0.7002 - loss: 0.8748 - val_accuracy: 0.7640 - val_loss: 0.6673 Epoch 37/50 104/104 - 6s - 54ms/step - accuracy: 0.7128 - loss: 0.8401 - val_accuracy: 0.7781 - val_loss: 0.6574 Epoch 38/50 104/104 - 4s - 40ms/step - accuracy: 0.7212 - loss: 0.8209 - val_accuracy: 0.8006 - val_loss: 0.5743 Epoch 39/50 104/104 - 7s - 65ms/step - accuracy: 0.7074 - loss: 0.8336 - val_accuracy: 0.7205 - val_loss: 0.7702 Epoch 40/50 104/104 - 4s - 40ms/step - accuracy: 0.7200 - loss: 0.8153 - val_accuracy: 0.7008 - val_loss: 0.8896 Epoch 41/50 104/104 - 4s - 38ms/step - accuracy: 0.7305 - loss: 0.8041 - val_accuracy: 0.7289 - val_loss: 0.7188 Epoch 42/50 104/104 - 5s - 49ms/step - accuracy: 0.7341 - loss: 0.7577 - val_accuracy: 0.7781 - val_loss: 0.6198 Epoch 43/50 104/104 - 5s - 46ms/step - accuracy: 0.7417 - loss: 0.7715 - val_accuracy: 0.7500 - val_loss: 0.7079 Epoch 44/50 104/104 - 4s - 40ms/step - accuracy: 0.7486 - loss: 0.7428 - val_accuracy: 0.8174 - val_loss: 0.5114 Epoch 45/50 104/104 - 5s - 45ms/step - accuracy: 0.7507 - loss: 0.7442 - val_accuracy: 0.7233 - val_loss: 0.7640 Epoch 46/50 104/104 - 5s - 49ms/step - accuracy: 0.7438 - loss: 0.7523 - val_accuracy: 0.8188 - val_loss: 0.5182 Epoch 47/50 104/104 - 4s - 37ms/step - accuracy: 0.7609 - loss: 0.7099 - val_accuracy: 0.7781 - val_loss: 0.6114 Epoch 48/50 104/104 - 7s - 65ms/step - accuracy: 0.7654 - loss: 0.7031 - val_accuracy: 0.8315 - val_loss: 0.4837 Epoch 49/50 104/104 - 9s - 84ms/step - accuracy: 0.7693 - loss: 0.6529 - val_accuracy: 0.7542 - val_loss: 0.7448 Epoch 50/50 104/104 - 7s - 66ms/step - accuracy: 0.7814 - loss: 0.6502 - val_accuracy: 0.7893 - val_loss: 0.5601
plot_loss(history3)
plot_accuracy(history3)
plot_confusion_matrix_and_report(model3, x_val_normalized, y_val_encoded)
23/23 ━━━━━━━━━━━━━━━━━━━━ 1s 18ms/step
precision recall f1-score support
0 0.43 0.80 0.56 40
1 0.81 0.98 0.89 59
2 0.97 0.79 0.87 43
3 0.98 0.90 0.94 91
4 0.67 0.85 0.75 33
5 0.85 0.96 0.90 71
6 0.98 0.53 0.69 98
7 0.69 0.94 0.79 33
8 0.88 0.49 0.63 77
9 0.40 0.66 0.50 35
10 0.93 0.96 0.95 74
11 0.90 0.78 0.83 58
accuracy 0.79 712
macro avg 0.79 0.80 0.77 712
weighted avg 0.84 0.79 0.79 712
Observations¶
Loss and Accuracy Plots:
- Training loss steadily decreases, indicating the model is learning effectively from the augmented data. Training accuracy gradually increases, stabilizing around 79%, which shows good learning without overfitting.
- Validation loss follows a similar decreasing trend, with occasional spikes due to the augmented data variability. Validation accuracy stabilizes around ~79%, indicating significant improvement over previous models and good generalization.
Confusion Matrix:
- The predictions are better distributed across all 12 classes. Classes such as
Common Chickweed,Loose Silky-bent, andSmall-flowered Cranesbillshow significant improvement in correct classifications. - Some classes, such as
Loose Silky-bent, are still misclassified asBlack-grassorCommon Chickweed. - Underrepresented classes like
Shepherds PurseandScentless Mayweedshow moderate confusion with possibly closely classes.
Classification Report:
Precision: 0.79 (macro average), up from 0.68 in Model2.
Recall: 0.80 (macro average), up from 0.66 in Model2.
F1-score: 0.79 (macro average), showing balanced performance across classes.
Minority classes like
Cleavers(F1-score 0.87),Small-flowered Cranesbill(F1-score 0.95), andSugar beet(F1-score 0.83) show improvement. Most classes have F1-scores above 0.70, indicating consistent and balanced performance.
Improvements Made:
Data Augmentation: The use of
ImageDataGeneratorintroduced variability in the training data, leading to better generalization and improved validation accuracy.Regularization: Adding Dropout layers reduced overfitting and helped improve validation performance.
Complexity: Adding an extra dense layer and increasing the number of neurons allowed the model to capture more complex patterns.
Strengths:
- Generalization: Validation accuracy improved significantly to match training accuracy.
- Balanced Performance: The F1-scores across classes indicate that the model performs well for both majority and minority classes.
- Reduced Overfitting: The training and validation curves are more aligned, showing reduced overfitting compared to previous models.
Fourth Iteration¶
Improvement Options
- Increasing number of epoch to 75 to check if the model trains more on the training dataset.
- Use of
ReduceLROnPlateauwith a factor=0.1, patience=5 and min learning rate up to 1e-6 (0.000001) - Also intoducing early stopping if loss does not decrease over 10 epochs.
# Creating & Fitting DataGenerator
augmented_datagen = ImageDataGenerator(
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True
)
augmented_datagen.fit(x_train_normalized)
# Define ReduceLROnPlateau callback <-- Added New
reduce_lr = ReduceLROnPlateau(
monitor='val_loss', # Monitor validation loss
factor=0.1, # Reduce learning rate by a factor of 0.1
patience=5, # Wait 5 epochs for improvement
min_lr=1e-6, # Set a minimum learning rate
verbose=1 # Print learning rate changes
)
# Optionally, add EarlyStopping for better monitoring <-- Added New
early_stopping = EarlyStopping(
monitor='val_loss',
patience=10, # Stop training if no improvement for 10 epochs
restore_best_weights=True # Restore the best weights from training
)
# Building the fourth model
model4 = Sequential([
# First Conv2D layer
Conv2D(filters=32, kernel_size=(3, 3), activation='relu', padding='same', input_shape=(64, 64, 3)),
# Second Conv2D layer
Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same'),
# First MaxPooling layer
MaxPooling2D(pool_size=(2, 2)),
# Third Conv2D layer
Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same'),
# Fourth Conv2D layer
Conv2D(filters=128, kernel_size=(3, 3), activation='relu', padding='same'),
# Second MaxPooling layer
MaxPooling2D(pool_size=(2, 2)),
# First Dropout Layer
Dropout(0.25),
# BatchNormalization
BatchNormalization(),
# Flatten the output
Flatten(),
# First Dense layer with 16 neurons
Dense(16, activation='relu'),
# Second Dense layer with 32 neurons
Dense(32, activation='relu'),
# Second Dropout Layer
Dropout(0.25),
# Output layer with 12 neurons for 12 classes
Dense(12, activation='softmax')
])
# Compile the model
model4.compile(optimizer=Adam(learning_rate=1e-4),# Reduced learning_rate=0.0005
loss='categorical_crossentropy',
metrics=['accuracy'])
# Print the model summary
model4.summary()
Model: "sequential_1"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ conv2d_4 (Conv2D) │ (None, 64, 64, 32) │ 896 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_5 (Conv2D) │ (None, 64, 64, 64) │ 18,496 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_2 (MaxPooling2D) │ (None, 32, 32, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_6 (Conv2D) │ (None, 32, 32, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_7 (Conv2D) │ (None, 32, 32, 128) │ 73,856 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_3 (MaxPooling2D) │ (None, 16, 16, 128) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout_2 (Dropout) │ (None, 16, 16, 128) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ batch_normalization_1 │ (None, 16, 16, 128) │ 512 │ │ (BatchNormalization) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ flatten_1 (Flatten) │ (None, 32768) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_3 (Dense) │ (None, 16) │ 524,304 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_4 (Dense) │ (None, 32) │ 544 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout_3 (Dropout) │ (None, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_5 (Dense) │ (None, 12) │ 396 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 655,932 (2.50 MB)
Trainable params: 655,676 (2.50 MB)
Non-trainable params: 256 (1.00 KB)
# Resetting the environment default seed=1
reset_environment()
# Fit the model
history4 = model4.fit(
augmented_datagen.flow( # Data Augmentation
x_train_normalized,
y_train_encoded,
batch_size=32),
epochs=75, # <-- Added New
validation_data=(x_val_normalized,
y_val_encoded),
class_weight=class_weights_dict,
callbacks=[reduce_lr, early_stopping], # <-- Added New
verbose=2)
Epoch 1/75 104/104 - 14s - 131ms/step - accuracy: 0.0917 - loss: 2.4606 - val_accuracy: 0.1419 - val_loss: 2.4766 - learning_rate: 1.0000e-04 Epoch 2/75 104/104 - 13s - 127ms/step - accuracy: 0.1678 - loss: 2.2593 - val_accuracy: 0.2037 - val_loss: 2.4191 - learning_rate: 1.0000e-04 Epoch 3/75 104/104 - 5s - 49ms/step - accuracy: 0.2087 - loss: 2.1042 - val_accuracy: 0.2584 - val_loss: 2.3243 - learning_rate: 1.0000e-04 Epoch 4/75 104/104 - 7s - 63ms/step - accuracy: 0.2424 - loss: 2.0196 - val_accuracy: 0.2584 - val_loss: 2.2018 - learning_rate: 1.0000e-04 Epoch 5/75 104/104 - 9s - 89ms/step - accuracy: 0.2824 - loss: 1.9240 - val_accuracy: 0.3455 - val_loss: 1.9642 - learning_rate: 1.0000e-04 Epoch 6/75 104/104 - 5s - 50ms/step - accuracy: 0.3068 - loss: 1.8605 - val_accuracy: 0.3961 - val_loss: 1.7538 - learning_rate: 1.0000e-04 Epoch 7/75 104/104 - 4s - 37ms/step - accuracy: 0.3528 - loss: 1.7715 - val_accuracy: 0.4537 - val_loss: 1.5667 - learning_rate: 1.0000e-04 Epoch 8/75 104/104 - 6s - 58ms/step - accuracy: 0.3456 - loss: 1.7474 - val_accuracy: 0.4270 - val_loss: 1.5738 - learning_rate: 1.0000e-04 Epoch 9/75 104/104 - 5s - 46ms/step - accuracy: 0.3660 - loss: 1.7005 - val_accuracy: 0.4831 - val_loss: 1.4819 - learning_rate: 1.0000e-04 Epoch 10/75 104/104 - 4s - 41ms/step - accuracy: 0.4012 - loss: 1.6588 - val_accuracy: 0.4621 - val_loss: 1.4873 - learning_rate: 1.0000e-04 Epoch 11/75 104/104 - 7s - 64ms/step - accuracy: 0.3979 - loss: 1.6353 - val_accuracy: 0.5211 - val_loss: 1.3143 - learning_rate: 1.0000e-04 Epoch 12/75 104/104 - 9s - 84ms/step - accuracy: 0.4373 - loss: 1.5464 - val_accuracy: 0.5169 - val_loss: 1.3746 - learning_rate: 1.0000e-04 Epoch 13/75 104/104 - 7s - 66ms/step - accuracy: 0.4382 - loss: 1.5432 - val_accuracy: 0.5084 - val_loss: 1.2595 - learning_rate: 1.0000e-04 Epoch 14/75 104/104 - 4s - 38ms/step - accuracy: 0.4547 - loss: 1.4959 - val_accuracy: 0.5941 - val_loss: 1.1640 - learning_rate: 1.0000e-04 Epoch 15/75 104/104 - 4s - 38ms/step - accuracy: 0.4683 - loss: 1.4821 - val_accuracy: 0.5801 - val_loss: 1.1780 - learning_rate: 1.0000e-04 Epoch 16/75 104/104 - 6s - 61ms/step - accuracy: 0.4776 - loss: 1.4364 - val_accuracy: 0.6110 - val_loss: 1.1521 - learning_rate: 1.0000e-04 Epoch 17/75 104/104 - 4s - 38ms/step - accuracy: 0.4974 - loss: 1.3982 - val_accuracy: 0.5520 - val_loss: 1.1993 - learning_rate: 1.0000e-04 Epoch 18/75 104/104 - 6s - 58ms/step - accuracy: 0.4947 - loss: 1.4028 - val_accuracy: 0.4284 - val_loss: 1.5066 - learning_rate: 1.0000e-04 Epoch 19/75 104/104 - 5s - 46ms/step - accuracy: 0.5068 - loss: 1.3732 - val_accuracy: 0.6110 - val_loss: 1.0438 - learning_rate: 1.0000e-04 Epoch 20/75 104/104 - 4s - 41ms/step - accuracy: 0.5038 - loss: 1.3469 - val_accuracy: 0.5857 - val_loss: 1.0751 - learning_rate: 1.0000e-04 Epoch 21/75 104/104 - 5s - 45ms/step - accuracy: 0.5248 - loss: 1.3147 - val_accuracy: 0.6461 - val_loss: 1.0268 - learning_rate: 1.0000e-04 Epoch 22/75 104/104 - 5s - 47ms/step - accuracy: 0.5453 - loss: 1.2753 - val_accuracy: 0.6826 - val_loss: 0.9388 - learning_rate: 1.0000e-04 Epoch 23/75 104/104 - 4s - 37ms/step - accuracy: 0.5420 - loss: 1.2885 - val_accuracy: 0.6194 - val_loss: 1.0192 - learning_rate: 1.0000e-04 Epoch 24/75 104/104 - 4s - 43ms/step - accuracy: 0.5555 - loss: 1.2275 - val_accuracy: 0.5885 - val_loss: 1.1245 - learning_rate: 1.0000e-04 Epoch 25/75 104/104 - 5s - 47ms/step - accuracy: 0.5762 - loss: 1.1709 - val_accuracy: 0.6208 - val_loss: 1.0716 - learning_rate: 1.0000e-04 Epoch 26/75 104/104 - 4s - 39ms/step - accuracy: 0.5886 - loss: 1.1713 - val_accuracy: 0.5323 - val_loss: 1.3115 - learning_rate: 1.0000e-04 Epoch 27/75 104/104 - 7s - 63ms/step - accuracy: 0.5835 - loss: 1.1570 - val_accuracy: 0.6545 - val_loss: 0.9381 - learning_rate: 1.0000e-04 Epoch 28/75 104/104 - 4s - 41ms/step - accuracy: 0.5937 - loss: 1.1330 - val_accuracy: 0.6531 - val_loss: 0.9837 - learning_rate: 1.0000e-04 Epoch 29/75 104/104 - 4s - 38ms/step - accuracy: 0.6018 - loss: 1.1134 - val_accuracy: 0.6784 - val_loss: 0.7970 - learning_rate: 1.0000e-04 Epoch 30/75 104/104 - 5s - 45ms/step - accuracy: 0.6174 - loss: 1.0690 - val_accuracy: 0.5702 - val_loss: 1.2178 - learning_rate: 1.0000e-04 Epoch 31/75 104/104 - 5s - 46ms/step - accuracy: 0.6223 - loss: 1.0510 - val_accuracy: 0.5843 - val_loss: 1.1018 - learning_rate: 1.0000e-04 Epoch 32/75 104/104 - 4s - 37ms/step - accuracy: 0.6268 - loss: 1.0309 - val_accuracy: 0.5871 - val_loss: 1.1697 - learning_rate: 1.0000e-04 Epoch 33/75 104/104 - 7s - 65ms/step - accuracy: 0.6346 - loss: 1.0215 - val_accuracy: 0.7261 - val_loss: 0.7669 - learning_rate: 1.0000e-04 Epoch 34/75 104/104 - 9s - 83ms/step - accuracy: 0.6508 - loss: 0.9679 - val_accuracy: 0.6882 - val_loss: 0.7650 - learning_rate: 1.0000e-04 Epoch 35/75 104/104 - 7s - 64ms/step - accuracy: 0.6547 - loss: 0.9858 - val_accuracy: 0.4803 - val_loss: 1.5556 - learning_rate: 1.0000e-04 Epoch 36/75 104/104 - 4s - 38ms/step - accuracy: 0.6466 - loss: 0.9770 - val_accuracy: 0.7654 - val_loss: 0.6657 - learning_rate: 1.0000e-04 Epoch 37/75 104/104 - 5s - 52ms/step - accuracy: 0.6547 - loss: 0.9424 - val_accuracy: 0.6671 - val_loss: 0.8831 - learning_rate: 1.0000e-04 Epoch 38/75 104/104 - 5s - 50ms/step - accuracy: 0.6776 - loss: 0.8883 - val_accuracy: 0.6685 - val_loss: 0.8156 - learning_rate: 1.0000e-04 Epoch 39/75 104/104 - 10s - 95ms/step - accuracy: 0.6842 - loss: 0.9033 - val_accuracy: 0.7640 - val_loss: 0.6301 - learning_rate: 1.0000e-04 Epoch 40/75 104/104 - 5s - 47ms/step - accuracy: 0.6899 - loss: 0.8876 - val_accuracy: 0.7149 - val_loss: 0.7230 - learning_rate: 1.0000e-04 Epoch 41/75 104/104 - 4s - 39ms/step - accuracy: 0.6860 - loss: 0.8815 - val_accuracy: 0.6138 - val_loss: 1.0689 - learning_rate: 1.0000e-04 Epoch 42/75 104/104 - 4s - 42ms/step - accuracy: 0.6989 - loss: 0.8371 - val_accuracy: 0.7528 - val_loss: 0.6694 - learning_rate: 1.0000e-04 Epoch 43/75 104/104 - 5s - 52ms/step - accuracy: 0.6929 - loss: 0.8434 - val_accuracy: 0.6812 - val_loss: 0.8303 - learning_rate: 1.0000e-04 Epoch 44/75 104/104 - 4s - 41ms/step - accuracy: 0.7005 - loss: 0.8352 - val_accuracy: 0.7753 - val_loss: 0.5881 - learning_rate: 1.0000e-04 Epoch 45/75 104/104 - 7s - 64ms/step - accuracy: 0.7221 - loss: 0.7982 - val_accuracy: 0.5590 - val_loss: 1.6689 - learning_rate: 1.0000e-04 Epoch 46/75 104/104 - 9s - 84ms/step - accuracy: 0.6941 - loss: 0.8216 - val_accuracy: 0.7514 - val_loss: 0.6738 - learning_rate: 1.0000e-04 Epoch 47/75 104/104 - 7s - 67ms/step - accuracy: 0.7149 - loss: 0.7897 - val_accuracy: 0.6292 - val_loss: 1.0360 - learning_rate: 1.0000e-04 Epoch 48/75 104/104 - 8s - 80ms/step - accuracy: 0.7311 - loss: 0.7722 - val_accuracy: 0.7963 - val_loss: 0.5191 - learning_rate: 1.0000e-04 Epoch 49/75 104/104 - 6s - 60ms/step - accuracy: 0.7290 - loss: 0.7600 - val_accuracy: 0.7907 - val_loss: 0.5765 - learning_rate: 1.0000e-04 Epoch 50/75 104/104 - 10s - 95ms/step - accuracy: 0.7447 - loss: 0.7273 - val_accuracy: 0.5997 - val_loss: 1.3149 - learning_rate: 1.0000e-04 Epoch 51/75 104/104 - 5s - 46ms/step - accuracy: 0.7380 - loss: 0.7436 - val_accuracy: 0.8020 - val_loss: 0.5507 - learning_rate: 1.0000e-04 Epoch 52/75 104/104 - 4s - 38ms/step - accuracy: 0.7326 - loss: 0.7532 - val_accuracy: 0.7795 - val_loss: 0.5919 - learning_rate: 1.0000e-04 Epoch 53/75 Epoch 53: ReduceLROnPlateau reducing learning rate to 9.999999747378752e-06. 104/104 - 5s - 43ms/step - accuracy: 0.7438 - loss: 0.7261 - val_accuracy: 0.7317 - val_loss: 0.6922 - learning_rate: 1.0000e-04 Epoch 54/75 104/104 - 5s - 48ms/step - accuracy: 0.7579 - loss: 0.6826 - val_accuracy: 0.8160 - val_loss: 0.4857 - learning_rate: 1.0000e-05 Epoch 55/75 104/104 - 4s - 38ms/step - accuracy: 0.7684 - loss: 0.6545 - val_accuracy: 0.8399 - val_loss: 0.4561 - learning_rate: 1.0000e-05 Epoch 56/75 104/104 - 4s - 40ms/step - accuracy: 0.7729 - loss: 0.6492 - val_accuracy: 0.8399 - val_loss: 0.4441 - learning_rate: 1.0000e-05 Epoch 57/75 104/104 - 6s - 53ms/step - accuracy: 0.7675 - loss: 0.6478 - val_accuracy: 0.8441 - val_loss: 0.4358 - learning_rate: 1.0000e-05 Epoch 58/75 104/104 - 4s - 37ms/step - accuracy: 0.7618 - loss: 0.6363 - val_accuracy: 0.8455 - val_loss: 0.4489 - learning_rate: 1.0000e-05 Epoch 59/75 104/104 - 4s - 37ms/step - accuracy: 0.7627 - loss: 0.6485 - val_accuracy: 0.8497 - val_loss: 0.4266 - learning_rate: 1.0000e-05 Epoch 60/75 104/104 - 6s - 61ms/step - accuracy: 0.7609 - loss: 0.6454 - val_accuracy: 0.8441 - val_loss: 0.4372 - learning_rate: 1.0000e-05 Epoch 61/75 104/104 - 4s - 37ms/step - accuracy: 0.7645 - loss: 0.6509 - val_accuracy: 0.8427 - val_loss: 0.4529 - learning_rate: 1.0000e-05 Epoch 62/75 104/104 - 4s - 39ms/step - accuracy: 0.7823 - loss: 0.6236 - val_accuracy: 0.8244 - val_loss: 0.4541 - learning_rate: 1.0000e-05 Epoch 63/75 104/104 - 6s - 55ms/step - accuracy: 0.7720 - loss: 0.6472 - val_accuracy: 0.8455 - val_loss: 0.4310 - learning_rate: 1.0000e-05 Epoch 64/75 Epoch 64: ReduceLROnPlateau reducing learning rate to 1e-06. 104/104 - 4s - 42ms/step - accuracy: 0.7723 - loss: 0.6182 - val_accuracy: 0.8385 - val_loss: 0.4274 - learning_rate: 1.0000e-05 Epoch 65/75 104/104 - 5s - 46ms/step - accuracy: 0.7780 - loss: 0.6113 - val_accuracy: 0.8329 - val_loss: 0.4286 - learning_rate: 1.0000e-06 Epoch 66/75 104/104 - 5s - 46ms/step - accuracy: 0.7826 - loss: 0.6115 - val_accuracy: 0.8371 - val_loss: 0.4310 - learning_rate: 1.0000e-06 Epoch 67/75 104/104 - 4s - 41ms/step - accuracy: 0.7768 - loss: 0.6144 - val_accuracy: 0.8441 - val_loss: 0.4272 - learning_rate: 1.0000e-06 Epoch 68/75 104/104 - 7s - 67ms/step - accuracy: 0.7747 - loss: 0.6379 - val_accuracy: 0.8455 - val_loss: 0.4261 - learning_rate: 1.0000e-06 Epoch 69/75 104/104 - 4s - 38ms/step - accuracy: 0.7829 - loss: 0.6120 - val_accuracy: 0.8427 - val_loss: 0.4294 - learning_rate: 1.0000e-06 Epoch 70/75 104/104 - 4s - 38ms/step - accuracy: 0.7814 - loss: 0.5919 - val_accuracy: 0.8399 - val_loss: 0.4290 - learning_rate: 1.0000e-06 Epoch 71/75 104/104 - 7s - 64ms/step - accuracy: 0.7838 - loss: 0.6242 - val_accuracy: 0.8357 - val_loss: 0.4380 - learning_rate: 1.0000e-06 Epoch 72/75 104/104 - 4s - 37ms/step - accuracy: 0.7693 - loss: 0.6260 - val_accuracy: 0.8469 - val_loss: 0.4340 - learning_rate: 1.0000e-06 Epoch 73/75 104/104 - 5s - 49ms/step - accuracy: 0.7826 - loss: 0.6275 - val_accuracy: 0.8413 - val_loss: 0.4287 - learning_rate: 1.0000e-06 Epoch 74/75 104/104 - 6s - 56ms/step - accuracy: 0.7814 - loss: 0.6244 - val_accuracy: 0.8427 - val_loss: 0.4289 - learning_rate: 1.0000e-06 Epoch 75/75 104/104 - 4s - 37ms/step - accuracy: 0.7844 - loss: 0.6103 - val_accuracy: 0.8413 - val_loss: 0.4293 - learning_rate: 1.0000e-06
plot_loss(history4)
plot_accuracy(history4)
plot_confusion_matrix_and_report(model4, x_val_normalized, y_val_encoded)
23/23 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step
precision recall f1-score support
0 0.49 0.75 0.59 40
1 0.97 0.95 0.96 59
2 0.84 0.95 0.89 43
3 0.97 0.92 0.94 91
4 0.88 0.88 0.88 33
5 0.95 0.85 0.90 71
6 0.90 0.62 0.73 98
7 0.82 1.00 0.90 33
8 0.83 0.81 0.82 77
9 0.66 0.77 0.71 35
10 0.97 0.91 0.94 74
11 0.76 0.90 0.83 58
accuracy 0.85 712
macro avg 0.84 0.86 0.84 712
weighted avg 0.86 0.85 0.85 712
Observations¶
Loss and Accuracy Plots
- The loss plot shows consistent reduction in training.
- Validation losses over the epochs. There are fluctuations in validation loss due to data augmentation but a general downward trend.
- The accuracy plot indicates that training accuracy has plateaued around 78%. with training accuracy lagging behind validation accuracy.
- Validation accuracy has plateaued around 85%, with training accuracy lagging slightly, suggesting a well-balanced model.
Confusion Matrix
- Misclassifications still occur, particularly among visually similar classes such as Loose
Silky-bentand other similar grasses. - Some minority classes like
Shepherds Pursestill show limited improvement. Class imbalance a possible reason.
Classification Report
Macro average precision, recall, and F1 score have improved significantly to 84-86%, with the weighted average closely matching.
Precision and recall are high for some dominant classes (e.g.,
Charlock,Common Chickweed,Cleavers), but lower for a few minority classes.
Improvements Made
Use of
ReduceLROnPlateau: Dynamically adjusting the learning rate helped stabilize training, leading to smoother convergence for both training and validation loss function.Increased Epochs: Increasing epochs to 75 allowed the model to learn and refine patterns more, resulting improved generalization.
Regularization: Adding dropout layers reduced overfitting, making validation metrics more similar with training metrics.
Better Data Augmentation: Augmented data helped robustness and variability, resulting in better generalization to unseen data.
Strengths
- Generalization: Training and validation loss/accuracy are well-aligned, indicating less overfitting.
- Class Performance: Improved precision, recall, and F1-scores across most classes.
- Stability: The use of
ReduceLROnPlateauensured stable convergence, with the model responding well to learning rate adjustments.
Fifth Iteration - Using Transfer Learning¶
Improvement Criteria
- Using pre-trained model -
tensorflow.keras.applications.VGG16 - This model can accept our reduced image size (64 X 64 X 3).
- Early stopping paitence increased to 20 from 10.
# The VGG16 model with 64 X 64 X 3 input size
base_model = VGG16(weights='imagenet', include_top=False, input_shape=(64, 64, 3))
base_model.summary()
Model: "vgg16"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ input_layer_7 (InputLayer) │ (None, 64, 64, 3) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block1_conv1 (Conv2D) │ (None, 64, 64, 64) │ 1,792 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block1_conv2 (Conv2D) │ (None, 64, 64, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block1_pool (MaxPooling2D) │ (None, 32, 32, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block2_conv1 (Conv2D) │ (None, 32, 32, 128) │ 73,856 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block2_conv2 (Conv2D) │ (None, 32, 32, 128) │ 147,584 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block2_pool (MaxPooling2D) │ (None, 16, 16, 128) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block3_conv1 (Conv2D) │ (None, 16, 16, 256) │ 295,168 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block3_conv2 (Conv2D) │ (None, 16, 16, 256) │ 590,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block3_conv3 (Conv2D) │ (None, 16, 16, 256) │ 590,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block3_pool (MaxPooling2D) │ (None, 8, 8, 256) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block4_conv1 (Conv2D) │ (None, 8, 8, 512) │ 1,180,160 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block4_conv2 (Conv2D) │ (None, 8, 8, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block4_conv3 (Conv2D) │ (None, 8, 8, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block4_pool (MaxPooling2D) │ (None, 4, 4, 512) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block5_conv1 (Conv2D) │ (None, 4, 4, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block5_conv2 (Conv2D) │ (None, 4, 4, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block5_conv3 (Conv2D) │ (None, 4, 4, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block5_pool (MaxPooling2D) │ (None, 2, 2, 512) │ 0 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 14,714,688 (56.13 MB)
Trainable params: 14,714,688 (56.13 MB)
Non-trainable params: 0 (0.00 B)
# Getting only the conv layers for transfer learning.
transfer_layer = base_model.get_layer('block5_pool')
vgg_base_model = Model(inputs=base_model.input, outputs=transfer_layer.output)
# Freeze the base model layers to prevent training
for layer in vgg_base_model.layers:
layer.trainable = False
for layer in vgg_base_model.layers:
print(layer.name, layer.trainable)
# Define ReduceLROnPlateau callback
reduce_lr = ReduceLROnPlateau(
monitor='val_loss', # Monitor validation loss
factor=0.1, # Reduce learning rate by a factor of 0.1
patience=5, # Wait 5 epochs for improvement
min_lr=1e-6, # Set a minimum learning rate
verbose=1 # Print learning rate changes
)
# Optionally, add EarlyStopping for better monitoring
early_stopping = EarlyStopping(
monitor='val_loss',
patience=20, # Stop training if no improvement for 10 epochs
restore_best_weights=True # Restore the best weights from training
)
# Add custom layers on top
transfer_model = Sequential([
vgg_base_model, # Pre-trained VGG base model
Flatten(),
Dense(128, activation='relu'), # Fully connected layer
Dropout(0.25), # Dropout for regularization
Dense(64, activation='relu'), # Another dense layer
Dropout(0.25), # Another dropout
Dense(12, activation='softmax') # Output layer for 12 classes
])
# Compile the model
transfer_model.compile(
optimizer=Adam(learning_rate=1e-4), # Learning rate for transfer learning
loss='categorical_crossentropy',
metrics=['accuracy']
)
# Print model summary
transfer_model.summary()
input_layer_7 False block1_conv1 False block1_conv2 False block1_pool False block2_conv1 False block2_conv2 False block2_pool False block3_conv1 False block3_conv2 False block3_conv3 False block3_pool False block4_conv1 False block4_conv2 False block4_conv3 False block4_pool False block5_conv1 False block5_conv2 False block5_conv3 False block5_pool False
Model: "sequential_3"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ functional_6 (Functional) │ (None, 2, 2, 512) │ 14,714,688 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ flatten_3 (Flatten) │ (None, 2048) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_9 (Dense) │ (None, 128) │ 262,272 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout_6 (Dropout) │ (None, 128) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_10 (Dense) │ (None, 64) │ 8,256 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout_7 (Dropout) │ (None, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_11 (Dense) │ (None, 12) │ 780 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 14,985,996 (57.17 MB)
Trainable params: 271,308 (1.03 MB)
Non-trainable params: 14,714,688 (56.13 MB)
# Resetting the environment default seed=1
reset_environment()
# Train the transfer learning model
history_transfer = transfer_model.fit(
augmented_datagen.flow(
x_train_normalized,
y_train_encoded,
batch_size=32
),
validation_data=(x_val_normalized, y_val_encoded),
epochs=75,
callbacks=[reduce_lr, early_stopping],
class_weight=class_weights_dict,
verbose=2
)
Epoch 1/75 104/104 - 12s - 117ms/step - accuracy: 0.1158 - loss: 2.4987 - val_accuracy: 0.1798 - val_loss: 2.3882 - learning_rate: 1.0000e-04 Epoch 2/75 104/104 - 5s - 49ms/step - accuracy: 0.1555 - loss: 2.3911 - val_accuracy: 0.2542 - val_loss: 2.2776 - learning_rate: 1.0000e-04 Epoch 3/75 104/104 - 9s - 90ms/step - accuracy: 0.1973 - loss: 2.3091 - val_accuracy: 0.2669 - val_loss: 2.1859 - learning_rate: 1.0000e-04 Epoch 4/75 104/104 - 6s - 56ms/step - accuracy: 0.2162 - loss: 2.2269 - val_accuracy: 0.3118 - val_loss: 2.1070 - learning_rate: 1.0000e-04 Epoch 5/75 104/104 - 4s - 40ms/step - accuracy: 0.2478 - loss: 2.1669 - val_accuracy: 0.3413 - val_loss: 2.0284 - learning_rate: 1.0000e-04 Epoch 6/75 104/104 - 5s - 49ms/step - accuracy: 0.2406 - loss: 2.1274 - val_accuracy: 0.3399 - val_loss: 1.9810 - learning_rate: 1.0000e-04 Epoch 7/75 104/104 - 6s - 60ms/step - accuracy: 0.2638 - loss: 2.0496 - val_accuracy: 0.3975 - val_loss: 1.8712 - learning_rate: 1.0000e-04 Epoch 8/75 104/104 - 5s - 44ms/step - accuracy: 0.2950 - loss: 2.0157 - val_accuracy: 0.3666 - val_loss: 1.8441 - learning_rate: 1.0000e-04 Epoch 9/75 104/104 - 7s - 63ms/step - accuracy: 0.2878 - loss: 1.9764 - val_accuracy: 0.3975 - val_loss: 1.8001 - learning_rate: 1.0000e-04 Epoch 10/75 104/104 - 4s - 40ms/step - accuracy: 0.3095 - loss: 1.9463 - val_accuracy: 0.4157 - val_loss: 1.7545 - learning_rate: 1.0000e-04 Epoch 11/75 104/104 - 4s - 39ms/step - accuracy: 0.3284 - loss: 1.8969 - val_accuracy: 0.4031 - val_loss: 1.7337 - learning_rate: 1.0000e-04 Epoch 12/75 104/104 - 7s - 67ms/step - accuracy: 0.3206 - loss: 1.8967 - val_accuracy: 0.4452 - val_loss: 1.6995 - learning_rate: 1.0000e-04 Epoch 13/75 104/104 - 8s - 82ms/step - accuracy: 0.3269 - loss: 1.8562 - val_accuracy: 0.4551 - val_loss: 1.6407 - learning_rate: 1.0000e-04 Epoch 14/75 104/104 - 6s - 56ms/step - accuracy: 0.3408 - loss: 1.8594 - val_accuracy: 0.4494 - val_loss: 1.6358 - learning_rate: 1.0000e-04 Epoch 15/75 104/104 - 4s - 40ms/step - accuracy: 0.3498 - loss: 1.8173 - val_accuracy: 0.4649 - val_loss: 1.6140 - learning_rate: 1.0000e-04 Epoch 16/75 104/104 - 5s - 51ms/step - accuracy: 0.3573 - loss: 1.7997 - val_accuracy: 0.4635 - val_loss: 1.6053 - learning_rate: 1.0000e-04 Epoch 17/75 104/104 - 6s - 54ms/step - accuracy: 0.3609 - loss: 1.7597 - val_accuracy: 0.4775 - val_loss: 1.5519 - learning_rate: 1.0000e-04 Epoch 18/75 104/104 - 4s - 43ms/step - accuracy: 0.3627 - loss: 1.7649 - val_accuracy: 0.4831 - val_loss: 1.5399 - learning_rate: 1.0000e-04 Epoch 19/75 104/104 - 6s - 56ms/step - accuracy: 0.3693 - loss: 1.7438 - val_accuracy: 0.4958 - val_loss: 1.5258 - learning_rate: 1.0000e-04 Epoch 20/75 104/104 - 5s - 46ms/step - accuracy: 0.3844 - loss: 1.7119 - val_accuracy: 0.5014 - val_loss: 1.5010 - learning_rate: 1.0000e-04 Epoch 21/75 104/104 - 4s - 40ms/step - accuracy: 0.3826 - loss: 1.7092 - val_accuracy: 0.5056 - val_loss: 1.4931 - learning_rate: 1.0000e-04 Epoch 22/75 104/104 - 7s - 68ms/step - accuracy: 0.3838 - loss: 1.6856 - val_accuracy: 0.4888 - val_loss: 1.4709 - learning_rate: 1.0000e-04 Epoch 23/75 104/104 - 4s - 40ms/step - accuracy: 0.3877 - loss: 1.6796 - val_accuracy: 0.5169 - val_loss: 1.4594 - learning_rate: 1.0000e-04 Epoch 24/75 104/104 - 5s - 49ms/step - accuracy: 0.3931 - loss: 1.6685 - val_accuracy: 0.5070 - val_loss: 1.4655 - learning_rate: 1.0000e-04 Epoch 25/75 104/104 - 7s - 63ms/step - accuracy: 0.4048 - loss: 1.6482 - val_accuracy: 0.5478 - val_loss: 1.4161 - learning_rate: 1.0000e-04 Epoch 26/75 104/104 - 9s - 91ms/step - accuracy: 0.4183 - loss: 1.6279 - val_accuracy: 0.5197 - val_loss: 1.4174 - learning_rate: 1.0000e-04 Epoch 27/75 104/104 - 5s - 50ms/step - accuracy: 0.4208 - loss: 1.6107 - val_accuracy: 0.5056 - val_loss: 1.4268 - learning_rate: 1.0000e-04 Epoch 28/75 104/104 - 11s - 105ms/step - accuracy: 0.4144 - loss: 1.6096 - val_accuracy: 0.5197 - val_loss: 1.4056 - learning_rate: 1.0000e-04 Epoch 29/75 104/104 - 9s - 83ms/step - accuracy: 0.4226 - loss: 1.5920 - val_accuracy: 0.5309 - val_loss: 1.3774 - learning_rate: 1.0000e-04 Epoch 30/75 104/104 - 7s - 69ms/step - accuracy: 0.4250 - loss: 1.5726 - val_accuracy: 0.5421 - val_loss: 1.3825 - learning_rate: 1.0000e-04 Epoch 31/75 104/104 - 8s - 78ms/step - accuracy: 0.4147 - loss: 1.5713 - val_accuracy: 0.5112 - val_loss: 1.3635 - learning_rate: 1.0000e-04 Epoch 32/75 104/104 - 6s - 56ms/step - accuracy: 0.4352 - loss: 1.5608 - val_accuracy: 0.5309 - val_loss: 1.3568 - learning_rate: 1.0000e-04 Epoch 33/75 104/104 - 9s - 83ms/step - accuracy: 0.4403 - loss: 1.5352 - val_accuracy: 0.5211 - val_loss: 1.3660 - learning_rate: 1.0000e-04 Epoch 34/75 104/104 - 6s - 56ms/step - accuracy: 0.4316 - loss: 1.5437 - val_accuracy: 0.5435 - val_loss: 1.3548 - learning_rate: 1.0000e-04 Epoch 35/75 104/104 - 9s - 83ms/step - accuracy: 0.4337 - loss: 1.5409 - val_accuracy: 0.5225 - val_loss: 1.3677 - learning_rate: 1.0000e-04 Epoch 36/75 104/104 - 6s - 56ms/step - accuracy: 0.4454 - loss: 1.5211 - val_accuracy: 0.5309 - val_loss: 1.3231 - learning_rate: 1.0000e-04 Epoch 37/75 104/104 - 4s - 40ms/step - accuracy: 0.4418 - loss: 1.5244 - val_accuracy: 0.5365 - val_loss: 1.3325 - learning_rate: 1.0000e-04 Epoch 38/75 104/104 - 5s - 46ms/step - accuracy: 0.4490 - loss: 1.4975 - val_accuracy: 0.5604 - val_loss: 1.3006 - learning_rate: 1.0000e-04 Epoch 39/75 104/104 - 5s - 51ms/step - accuracy: 0.4611 - loss: 1.4805 - val_accuracy: 0.5365 - val_loss: 1.3030 - learning_rate: 1.0000e-04 Epoch 40/75 104/104 - 4s - 41ms/step - accuracy: 0.4671 - loss: 1.4825 - val_accuracy: 0.5351 - val_loss: 1.3010 - learning_rate: 1.0000e-04 Epoch 41/75 104/104 - 5s - 48ms/step - accuracy: 0.4544 - loss: 1.4808 - val_accuracy: 0.5478 - val_loss: 1.2800 - learning_rate: 1.0000e-04 Epoch 42/75 104/104 - 5s - 49ms/step - accuracy: 0.4701 - loss: 1.4619 - val_accuracy: 0.5534 - val_loss: 1.2805 - learning_rate: 1.0000e-04 Epoch 43/75 104/104 - 4s - 41ms/step - accuracy: 0.4614 - loss: 1.4665 - val_accuracy: 0.5548 - val_loss: 1.2666 - learning_rate: 1.0000e-04 Epoch 44/75 104/104 - 5s - 50ms/step - accuracy: 0.4668 - loss: 1.4623 - val_accuracy: 0.5506 - val_loss: 1.2632 - learning_rate: 1.0000e-04 Epoch 45/75 104/104 - 5s - 46ms/step - accuracy: 0.4743 - loss: 1.4342 - val_accuracy: 0.5492 - val_loss: 1.2737 - learning_rate: 1.0000e-04 Epoch 46/75 104/104 - 4s - 39ms/step - accuracy: 0.4728 - loss: 1.4289 - val_accuracy: 0.5632 - val_loss: 1.2551 - learning_rate: 1.0000e-04 Epoch 47/75 104/104 - 5s - 51ms/step - accuracy: 0.4767 - loss: 1.4264 - val_accuracy: 0.5365 - val_loss: 1.2706 - learning_rate: 1.0000e-04 Epoch 48/75 104/104 - 9s - 88ms/step - accuracy: 0.4890 - loss: 1.4150 - val_accuracy: 0.5534 - val_loss: 1.2386 - learning_rate: 1.0000e-04 Epoch 49/75 104/104 - 7s - 65ms/step - accuracy: 0.4941 - loss: 1.4036 - val_accuracy: 0.5435 - val_loss: 1.2637 - learning_rate: 1.0000e-04 Epoch 50/75 104/104 - 9s - 82ms/step - accuracy: 0.4842 - loss: 1.3975 - val_accuracy: 0.5548 - val_loss: 1.2461 - learning_rate: 1.0000e-04 Epoch 51/75 104/104 - 6s - 57ms/step - accuracy: 0.4968 - loss: 1.3976 - val_accuracy: 0.5604 - val_loss: 1.2417 - learning_rate: 1.0000e-04 Epoch 52/75 104/104 - 8s - 81ms/step - accuracy: 0.4800 - loss: 1.4062 - val_accuracy: 0.5449 - val_loss: 1.2548 - learning_rate: 1.0000e-04 Epoch 53/75 104/104 - 6s - 60ms/step - accuracy: 0.4659 - loss: 1.4143 - val_accuracy: 0.5871 - val_loss: 1.2123 - learning_rate: 1.0000e-04 Epoch 54/75 104/104 - 10s - 100ms/step - accuracy: 0.4845 - loss: 1.3733 - val_accuracy: 0.5590 - val_loss: 1.2214 - learning_rate: 1.0000e-04 Epoch 55/75 104/104 - 4s - 42ms/step - accuracy: 0.4959 - loss: 1.3747 - val_accuracy: 0.5702 - val_loss: 1.2212 - learning_rate: 1.0000e-04 Epoch 56/75 104/104 - 4s - 39ms/step - accuracy: 0.4887 - loss: 1.3853 - val_accuracy: 0.5435 - val_loss: 1.2523 - learning_rate: 1.0000e-04 Epoch 57/75 104/104 - 7s - 66ms/step - accuracy: 0.4866 - loss: 1.3902 - val_accuracy: 0.5716 - val_loss: 1.2178 - learning_rate: 1.0000e-04 Epoch 58/75 104/104 - 4s - 39ms/step - accuracy: 0.5098 - loss: 1.3549 - val_accuracy: 0.5772 - val_loss: 1.2007 - learning_rate: 1.0000e-04 Epoch 59/75 104/104 - 5s - 49ms/step - accuracy: 0.5050 - loss: 1.3666 - val_accuracy: 0.5702 - val_loss: 1.2090 - learning_rate: 1.0000e-04 Epoch 60/75 104/104 - 6s - 57ms/step - accuracy: 0.5056 - loss: 1.3416 - val_accuracy: 0.5801 - val_loss: 1.2075 - learning_rate: 1.0000e-04 Epoch 61/75 104/104 - 8s - 81ms/step - accuracy: 0.4956 - loss: 1.3554 - val_accuracy: 0.5843 - val_loss: 1.2052 - learning_rate: 1.0000e-04 Epoch 62/75 104/104 - 6s - 57ms/step - accuracy: 0.5149 - loss: 1.3350 - val_accuracy: 0.5758 - val_loss: 1.1751 - learning_rate: 1.0000e-04 Epoch 63/75 104/104 - 9s - 83ms/step - accuracy: 0.5041 - loss: 1.3400 - val_accuracy: 0.5997 - val_loss: 1.1639 - learning_rate: 1.0000e-04 Epoch 64/75 104/104 - 6s - 56ms/step - accuracy: 0.5101 - loss: 1.3255 - val_accuracy: 0.5730 - val_loss: 1.1950 - learning_rate: 1.0000e-04 Epoch 65/75 104/104 - 9s - 84ms/step - accuracy: 0.5083 - loss: 1.3367 - val_accuracy: 0.5660 - val_loss: 1.1882 - learning_rate: 1.0000e-04 Epoch 66/75 104/104 - 6s - 55ms/step - accuracy: 0.5035 - loss: 1.3278 - val_accuracy: 0.5772 - val_loss: 1.1780 - learning_rate: 1.0000e-04 Epoch 67/75 104/104 - 9s - 88ms/step - accuracy: 0.5098 - loss: 1.3332 - val_accuracy: 0.5772 - val_loss: 1.1861 - learning_rate: 1.0000e-04 Epoch 68/75 Epoch 68: ReduceLROnPlateau reducing learning rate to 9.999999747378752e-06. 104/104 - 6s - 54ms/step - accuracy: 0.5140 - loss: 1.3124 - val_accuracy: 0.5829 - val_loss: 1.1645 - learning_rate: 1.0000e-04 Epoch 69/75 104/104 - 4s - 40ms/step - accuracy: 0.5278 - loss: 1.3108 - val_accuracy: 0.5871 - val_loss: 1.1592 - learning_rate: 1.0000e-05 Epoch 70/75 104/104 - 7s - 63ms/step - accuracy: 0.5269 - loss: 1.2925 - val_accuracy: 0.5927 - val_loss: 1.1600 - learning_rate: 1.0000e-05 Epoch 71/75 104/104 - 9s - 85ms/step - accuracy: 0.5236 - loss: 1.3082 - val_accuracy: 0.5885 - val_loss: 1.1646 - learning_rate: 1.0000e-05 Epoch 72/75 104/104 - 7s - 66ms/step - accuracy: 0.5137 - loss: 1.2952 - val_accuracy: 0.5871 - val_loss: 1.1623 - learning_rate: 1.0000e-05 Epoch 73/75 104/104 - 8s - 81ms/step - accuracy: 0.5266 - loss: 1.2897 - val_accuracy: 0.5885 - val_loss: 1.1640 - learning_rate: 1.0000e-05 Epoch 74/75 Epoch 74: ReduceLROnPlateau reducing learning rate to 1e-06. 104/104 - 6s - 58ms/step - accuracy: 0.5215 - loss: 1.3048 - val_accuracy: 0.5885 - val_loss: 1.1649 - learning_rate: 1.0000e-05 Epoch 75/75 104/104 - 4s - 39ms/step - accuracy: 0.5272 - loss: 1.2770 - val_accuracy: 0.5899 - val_loss: 1.1640 - learning_rate: 1.0000e-06
plot_loss(history_transfer)
plot_accuracy(history_transfer)
Observations & Improvements
plot_confusion_matrix_and_report(transfer_model, x_val_normalized, y_val_encoded)
23/23 ━━━━━━━━━━━━━━━━━━━━ 1s 32ms/step
precision recall f1-score support
0 0.34 0.60 0.43 40
1 0.59 0.58 0.58 59
2 0.89 0.72 0.79 43
3 0.78 0.66 0.71 91
4 0.52 0.45 0.48 33
5 0.57 0.45 0.50 71
6 0.75 0.63 0.69 98
7 0.49 0.76 0.60 33
8 0.72 0.44 0.55 77
9 0.35 0.37 0.36 35
10 0.61 0.69 0.65 74
11 0.44 0.64 0.52 58
accuracy 0.59 712
macro avg 0.59 0.58 0.57 712
weighted avg 0.62 0.59 0.59 712
Observations¶
Loss and Accuracy Plots
- The model shows gradual improvement in training accuracy.
- The validation accuracy converges around 59%.
- Both training and validation loss are decreasing, which indicates proper learning without significant overfitting.
Confusion Matrix:
- The model struggles in certain classes (e.g.,
Loose Silky-bentandShepherd's Purse), with low recall and precision in some underrepresented categories. - Overall Accuracy 59% indicates the model is not fully leveraging the power of the VGG16 base model.
Classification Report
- Macro average precision, recall, and F1 score is not close to previous two models.
- Precision and recall are also not as high as previous model but moderately OK.
All this suggests that the model might not be adequately trained or lacks capacity for this dataset.
Comparison Between Model4 (Custom CNN) and Transfer_Model (VGG16 Transfer Learning)¶
Model4:
- Custom-built CNN with relatively fewer convolutional and dense layers. Includes Dropout, BatchNormalization, and dense layers with smaller capacities.
- Training Accuracy: 81%
- Validation Accuracy: 85%
- Custom model outperforms transfer learning in this specific task, especially in validation accuracy.
- Shows better generalization as validation accuracy is higher than training accuracy.
- Slightly Computationally efficient
Transfer_Model:
- Utilizes the pre-trained VGG16 architecture, having its deep and hierarchical feature extraction. Pre-trained on ImageNet.
- Training Accuracy: 59%
- Validation Accuracy: 59%
- Limited performance possibly due to frozen VGG16 layers and smaller input size (64x64x3).
- Issues with generalization on this dataset, likely due to not being fine-tuned for the specific task.
- Took almost similar training time, even with the frozen layers.
Key Takeaway¶
- Model4 is better suited for this specific dataset, offering higher accuracy and better generalization with lower computational cost.
- Transfer_Model underperforms in its current setup, but fine-tuning, higher input resolution, or using a different pre-trained model may improve its performance.
Final Model¶
Brief Comparison of All 5 Models:¶
Model1 (Base CNN):
- Performance: Poor generalization with accuracy around 38%; struggled to extract meaningful features.
Model2 (Enhanced CNN):
- Improvement: Added BatchNormalization and additional Conv2D layers; achieved better accuracy (~70%) but still limited in learning complexity.
Model3 (Data Augmentation):
- Impact: Data augmentation and Dropout improved generalization; reached ~79% accuracy, indicating robustness to overfitting.
Model4 (Optimized CNN):
- Best Performer: Further tuning with ReduceLROnPlateau and EarlyStopping resulted in ~85% accuracy, balancing generalization and learning.
Transfer_Model (VGG16):
- Underperformed: Achieved ~59% accuracy; limited by frozen layers and smaller input size, highlighting the need for fine-tuning for better results.
Final Model Selection¶
I would choose Model4 (Optimized Custom CNN) as the final model because:
- Highest Accuracy: It achieved an overall accuracy of 85%, outperforming all other models in both generalization and learning ability.
- Balanced Performance: The combination of BatchNormalization, Dropout, ReduceLROnPlateau, and EarlyStopping effectively prevented overfitting while optimizing training.
- Scalability: The architecture can be scaled or further tuned (e.g., by increasing input size or adding more layers) if needed, providing flexibility for future improvements.
Thus, Model4 provides the best trade-off between accuracy, computational efficiency, and adaptability for this specific task.
Performance of Final model (Model-4) on test data¶
plot_confusion_matrix_and_report(model4, x_test_normalized, y_test_encoded)
23/23 ━━━━━━━━━━━━━━━━━━━━ 1s 40ms/step
precision recall f1-score support
0 0.50 0.67 0.57 39
1 0.82 0.93 0.87 58
2 0.83 0.91 0.87 43
3 0.93 0.95 0.94 92
4 0.81 0.88 0.84 33
5 0.91 0.81 0.85 72
6 0.90 0.57 0.70 98
7 0.71 0.97 0.82 33
8 0.89 0.90 0.89 78
9 0.79 0.76 0.78 34
10 0.94 0.84 0.89 75
11 0.69 0.81 0.75 58
accuracy 0.82 713
macro avg 0.81 0.83 0.81 713
weighted avg 0.84 0.82 0.82 713
Observations:¶
Multiclass Classification Confusion Matrix
- The matrix shows relatively fewer misclassifications, with most predictions concentrated along the diagonal.
- Most classes, such as
Cleavers,Common Chickweed, andFat Hen, exhibit strong performance with F1-scores above 80%. - Some classes, such as
Loose Silky-bentandShepherd's Purse, have misclassifications which indicates room for improvement in handling these categories.
PerformanceMetrics
- The model achieves an accuracy of 82%, indicating it can correctly classify the majority of test samples.
- Macro Precision: 81%, reflecting balanced precision across all classes.
- Macro Recall: 83%, showing good recall for minority classes as well.
- Macro F1-Score: 81%, highlighting consistent performance across all metrics.
Conclusion: The final model demonstrates strong generalization and consistent performance on the test dataset, validating its suitability for practical use.
Visualizing the prediction¶
# Visualizing the predicted and correct label of images from test data
def classify_image(index):
plt.figure(figsize=(2,2))
plt.imshow(x_test[index])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', label_binarizer.inverse_transform(model4.predict((x_test_normalized[index].reshape(1,64,64,3)))))
print('True Label', label_binarizer.inverse_transform(y_test_encoded)[index])
classify_image(2)
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 17ms/step Predicted Label ['Small-flowered Cranesbill'] True Label Small-flowered Cranesbill
classify_image(4)
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 17ms/step Predicted Label ['Fat Hen'] True Label Fat Hen
classify_image(10)
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 20ms/step Predicted Label ['Sugar beet'] True Label Sugar beet
classify_image(15)
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 17ms/step Predicted Label ['Maize'] True Label Shepherds Purse
classify_image(23)
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step Predicted Label ['Scentless Mayweed'] True Label Scentless Mayweed
classify_image(1)
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step Predicted Label ['Common Chickweed'] True Label Scentless Mayweed
classify_image(61)
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step Predicted Label ['Scentless Mayweed'] True Label Scentless Mayweed
Actionable Insights and Business Recommendations¶
Actionable Insights:¶
High-performing Classes: The model performs well on certain categories like
Common Chickweed,Cleavers, andFat Hen, with F1-scores above 80%. These can be trusted for reliable automation in classifying these plant species.Underperforming Classes: Classes such as
Loose Silky-bentandShepherd's Purseshow lower recall and precision, potential overlap in visual features or lack of sufficient training observations. Collecting more diverse samples for these classes may improve performance.Generalization and Robustness on Test Data: With an overall accuracy of 82% and balanced class-wise metrics, the model demonstrates good generalization.
Business Recommendations:¶
Deploy the Model for Automation: Leverage the model to automate the classification of plant seedlings, reducing manual labor costs and improving efficiency in agricultural operations. Focus on high-performing categories to build initial confidence in the system.
Improve Data for Specific Classes: Collect more high-quality labeled data for underperforming classes like
Loose Silky-bentandShepherd's Purse.Monitor and Enhance Over Time: A feedback loop where user corrections on misclassifications are incorporated will be benificial. It will ensures continuous improvement and adaptability to new data.
Explore Scalability Using Higher Resolution Images: Evaluate the model's scalability to larger image sizes if necessary for even finer-grained classification.